In image and video processing, it may be preferable for some algorithms to capture several shots of a given scene to improve or realise a specific, e.g. noise reduction process so as to obtain one image of the scene having enhanced quality.
These shots are often captured in different conditions, such as, for example, light conditions or exposure time, to fit with specific applications.
Digital images, namely images that have been captured and digitised take the form of a generally rectangular array of, for example digital, values. For a bi-dimensional image, the association of a point in the array and of grey levels or colour levels, for instance red, blue and green levels in a RGB colour space, is called a pixel. Pixels may be likened to points on a grid having a corresponding grey level value or corresponding colour level values. For example, the grey level values or colour level values at any given pixel can be either an integer or a real number.
Each point is the result of a measurement that can be made by a matrix or a strip of light sensors such as couple charge devices (CCDs) or a CMOS imaging device. In a small square of e.g. the CCD matrix corresponding to a point, the number of photons arriving in a given period of time is counted. The arrival of said photons follows a random process introducing fluctuations about a mean value. Each sensor may further produce noise which is superimposed on the photon count. Images obtained by other methods such as photosensitive paper printing, also exhibit noise caused by the e.g. chemical characteristics in the backing used; this noise is retained during the digitisation of the photograph.
Most digital images therefore contain noise as a random disturbance of the value of the points. The degree of noise can depend on the grey or colour levels of the point, and on the illumination conditions. For instance, it may happen that there is more noise in brighter parts of an image even if the signal-to-noise ratio is better than at other points of the same image.
A signal-to-noise ratio is used to designate the proportion of a grey or colour level that may originate from noise. Noise becomes detectable when a mean amplitude or standard deviation is exceeded. It is desirable to eliminate noise from images since it impairs the clarity and detail of images. However, if noise is removed with an average filter, the image may become noise free but totally blurry and not sharper, actually.
In processes requiring the use of algorithms, particularly so-called denoising algorithms, several shots may be captured of the same content in the same light conditions and used as basis for processing. Comparison of the captured shots aids in noise and detail discrimination.
In High Dynamic Range (HDR) algorithms, several shots may be captured with a different exposure time. Lightest and darkest areas of an image are revealed by adapting control of the exposure time to allow a greater dynamic range of luminescence.
The use of a flash in image capture increases image quality but can generate unwanted effects such as red eye or can result in an over exposed area. In red eye detection and elimination schemes, shots are taken with and without use of a flash. Comparison between the shots means the position of the eyes can be detected and the flash image can be used for HDR algorithms.
Because many denoising algorithms are unable to differentiate between noise and small details of the image, a number of shots may be used as a comparator to help distinguish between image detail and noise.
However, the content of multiple shots are seldom aligned due to, for example, user hand or scene movement between image acquisitions. The resulting mismatch is problematic for the data merge step. For that reason, some algorithms use stabilisation methods to align the data prior to data crop along the image border. However, the efficiency of such methods is correlated with their complexity and requires specific architecture. Further information regarding this correlation is described, for instance, by Auberger and Miro in “Digital video stabilization architecture for low cost devices”, Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005.
Furthermore, the capture and storing of multiple images requires additional memory to store the data of each shot, additional power during image capture and a means of traffic management between each frame captured. Additionally, the required stabilisation step is complicated by the absence of a standardised time lapse between each shot common to device manufacturers, with longer time lapse periods resulting in a greater difference between shots and a subsequent increased difficulty in stabilisation.